This disclosure provides a static occlusion handling method and system for use with appearance-based video tracking algorithms where static occlusions are present. The method and system assumes that the objects to be tracked move according to structured motion patterns within a scene, such as vehicles moving along a roadway. A primary concept is to replicate pixels associated with the tracked object from previous and current frames to current and future frames when the tracked object is occluded, partially or otherwise, by a semi-permanent object in the scene that hinders the view of the tracked object, where the predicted motion of the tracked object is a basis for replication of the pixels. In the context of this disclosure, the semi-permanent objects that interfere with a view of an object being tracked are termed ‘static occlusions’.
Static occlusions present challenges in surveillance applications such as person and vehicle tracking. In most presently available tracking algorithms, a track is defined for an object and the position of the object is updated on a frame-by-frame basis by matching known features and/or blob characteristics of the object as it traverses an area of interest. Tracking is typically performed by finding the location of the best-matching features given a current feature representation of the object being tracked. This is usually performed via search/optimization algorithms across regions in subsequent frames nearby the location of the object in the current frame. When these features and blobs are occluded by other objects, the track can be lost since the appearance of the occluded object may not sufficiently match the appearance of the unoccluded object. Known methods to recover these “lost” tracks are not adequate for many applications.
A Kalman filter method uses linear prediction to estimate future object locations based on a past trajectory. See Azari, M.; Seyfi, A.; Rezaie, A. H., “Real Time Multiple Object Tracking and Occlusion Reasoning Using Adaptive Kalman Filters”, Machine Vision and Image Processing (MVIP), 2011, 7th Iranian, pages 1-5, Nov. 16-17, 2011. However, a key drawback of the Kalman filter approach is the loss of the tracked object if the actual time the object is occluded differs from the filter prediction, e.g., when a car (tracked object) stops behind a tree (static occlusion). Other approaches analyze the dynamics of blob deformations, such as merging, splitting, shrinking, expanding, etc., that occur moments before the track is lost in order for it to potentially be recovered in future frames. See Jiyan Pan; Bo Hu, “Robust Occlusion Handling in Object Tracking”, Computer Vision and Pattern Recognition, 2007, CVPR '07, IEEE Conference, pages 1-8, Jun. 17-22, 2007; and Gwo-Cheng Chao; Shyh-Kang Jeng; Shung-Shing Lee, “An improved occlusion handling for appearance-based tracking”, Image Processing (ICIP), 2011 18th IEEE International Conference, pages 465-468, Sep. 11-14, 2011. Notably, these other approaches rely on computationally expensive search windows and morphological processing.